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Abstract 

We used a multi-component single-subject experimental design across three preschool teachers to examine the effects 
of video self-monitoring with graduated training and feedback on the accuracy with which teachers monitored their 
implementation of embedded instructional learning trials. We also examined changes in teachers’ implementation of 
learning trials. In each self-monitoring condition, teachers observed and recorded their implemented learning trials using 
video and a coding form. Conditions differed in the specificity of prompts on the coding form and the type of training and 
feedback provided. The combination of training, coding forms with specific prompts for learning trial components, and 
external feedback generally resulted in more accurate self-monitoring for two of three participants and increases in the 
fidelity of implementation of learning trials. Findings suggest self-monitoring can be effective for increasing the fidelity with 
which teachers implement embedded instructional learning trials, but systematic training and feedback are important for 


ensuring self-monitoring accuracy. 
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A fundamental indicator of quality instruction in preschool 
classrooms is accurate implementation of instructional 
learning trials embedded across naturally occurring activi- 
ties and routines to support children’s acquisition of indi- 
vidualized learning targets (Barton, Bishop, & Snyder, 
2014; Snyder, Hemmeter, McLean, Sandall, & McLaughlin, 
2013). Embedded instruction involves teachers implement- 
ing three-term contingency instructional learning trials. 
These trials include an adult or environmental antecedent 
(A) to set the occasion for the child to practice a targeted 
behavior, the child’s response (B) to the antecedent, and an 
adult- or environmentally delivered consequence (C) or 
error correction procedure immediately following the 
child’s behavior. 

Researchers have emphasized the importance of quanti- 
fying the accuracy of instructional learning trials as a basic 
unit of instruction (Albers & Greer, 1991; B. F. Skinner, 
1968; C. H. Skinner, Fletcher, & Hennington, 1996; 
VanDerHeyden, Snyder, Smith, Sevin, & Longwell, 2005; 
Vargus & Vargus, 1991). Complete learning trials (CLTs) is 
a term used to describe embedded learning trials in which 
the A and C components of the three-term contingency are 
implemented by a teacher around a selected child behavior 


(B) component (Barton et al., 2014; Snyder et al., 2013). 
Research has shown a positive relationship between the fre- 
quent implementation of CLTs and children’s engagement 
and learning (VanDerHeyden et al., 2005). Despite research 
supporting the importance of teachers understanding and 
implementing embedded instruction trials that involve the 
three-term contingency, obtaining fidelity of implementa- 
tion in preschool classrooms during ongoing activities has 
been challenging (McBride & Schwartz, 2003). 

One potential explanation for this lack of fidelity might 
be that embedding instructional trials requires careful plan- 
ning and support (Snyder et al., 2013). Teachers often have 
limited access to implementation supports that might help 
them understand the three-term contingency in relation to 
instructional trials or to evaluate their implementation of 
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CLTs (Barton et al., 2014). Implementation supports teach- 
ers might access include materials or assistance from exter- 
nal agents (e.g., training, mentoring, coaching) or use of 
internal processes (e.g., self-monitoring, self-reflection, 
self-coaching). 

Coaching is an implementation support demonstrated to 
be effective for improving teachers’ implementation of 
instructional practices (e.g., Fox, Hemmeter, Snyder, 
Binder, & Clarke, 2011; Hemmeter, Snyder, Kinder, & 
Artman, 2011). Coaching in many published studies has 
been delivered by an expert (Snyder et al., 2012). There is 
growing interest in identifying alternative delivery formats 
for effective coaching. Effective coaching includes oppor- 
tunities for goal-setting and action planning, focused obser- 
vation of teachers implementing evidence-based teaching 
practices, and reflection and feedback (National Center on 
Quality Teaching and Learning [NCQTL], 2014). 

Self-coaching is a format that includes self-guided goal- 
setting and self-monitoring of one’s implementation of 
effective teaching practices (NCQTL, 2014). Self- 
monitoring is characterized by self-observation and self- 
recording, in which an individual observes for a specific 
behavior, records the occurrence of that behavior, and self- 
evaluates regarding aspects of one’s performance (Kazdin, 
1994; Maag, 1999). Explicitly teaching self-monitoring 
might be an efficient strategy for supporting implementa- 
tion of effective teaching practices, including CLTs. 

Self-monitoring has been shown to be efficacious for 
improving teachers’ implementation of a variety of instruc- 
tional strategies, including instructional learning trials 
(Belfiore, Fritts, & Herman, 2008; Lylo & Lee, 2013; 
Pelletier, McNamara, Braga-Kenyon, & Ahearn, 2010), 
components of token economy interventions (Petscher & 
Bailey, 2006; Plavnick, Ferreri, & Maupin, 2010), and 
praise statements (Kalis, Vannest, & Parker, 2007; Wright, 
Ellis, & Baxter, 2012). Only two of these studies were con- 
ducted in early childhood settings (Plavnick et al., 2010; 
Wright et al., 2012). Methods used to support teachers’ self- 
monitoring ranged from checklists completed based on 
teacher recall (Petscher & Bailey, 2006; Plavnick et al., 
2010), in situ monitoring via hand-held counters (Kalis et 
al., 2007), and reviewing audio or video recordings and not- 
ing the occurrence of targeted teaching behaviors (Belfiore 
et al., 2008; Lylo & Lee, 2013; Pelletier et al., 2010). 
Although training to implement the self-monitoring inter- 
vention was included in every study, the intensity of train- 
ing varied based on the method used to self-monitor. 

Of particular note across these studies is the varied use of 
feedback procedures to enhance implementation of the self- 
monitoring intervention. In four of the seven studies (Belfiore 
et al., 2008; Lylo & Lee, 2013; Pelletier et al., 2010; Wright 
et al., 2012), feedback on participants’ self-monitoring 
accuracy was provided by a researcher during initial train- 
ing to implement the self-monitoring intervention but not 


after participants began self-monitoring independently. 
Two studies included feedback from a researcher on partici- 
pants’ self-monitoring accuracy during training and for the 
duration of the self-monitoring intervention (Kalis et al., 
2007; Petscher & Bailey, 2006), and one study did not 
include a feedback component (Plavnick et al., 2010). None 
of the studies examined relationships between variations in 
training and feedback on self-monitoring behaviors and the 
accuracy of self-monitoring. In addition, no studies exam- 
ined relationships between self-monitoring accuracy and 
participants’ implementation of the monitored instructional 
practices (e.g., components of the token economy, discrete 
learning trials). 

The purpose of the present study was to extend the 
research on self-monitoring as a component of self-coach- 
ing, with a focus on early childhood teachers’ instructional 
practices. We used a multi-component single-subject exper- 
imental design across three teachers to evaluate three video 
self-monitoring conditions. Each self-monitoring condition 
involved teachers viewing video of themselves implement- 
ing embedded instructional learning trials in their class- 
rooms and completing a self-monitoring coding form with 
prompts corresponding to the components of instructional 
learning trials (i.e., antecedent, behavior, consequence/error 
correction). Two research questions were addressed: 


Research Question 1: What are the functional relation- 
ships between graduated training and feedback and 
teachers’ accuracy of self-monitoring their embedded 
instruction learning trials? 

Research Question 2: How does the completeness of 
teachers’ learning trials (CLTs) change across the experi- 
mental conditions? 


Method 


Participants 


We recruited teachers working in an inclusive, university- 
based early care and education center who were interested in 
using video to self-monitor their instructional practices. To 
be eligible for the study, teachers had to (a) be the lead 
teacher of children between the ages of 24 and 60 months, 
(b) have at least one study-eligible child in their classroom, 
and (c) be able to view and code up to five, 10- to 20-min 
videos of themselves providing instructional trials to a child 
during a classroom activity, routine, or transition each week. 


Teacher participants. All three teachers were female and 
identified themselves as White/non-Hispanic. Natalie was 
24 years old when the study began. She held a bachelor’s 
degree in special education and a master’s degree in early 
childhood education, with a director’s credential and teach- 
ing endorsements in pre-kindergarten, pre-kindergarten 
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disabilities, and English for Speakers of Other Languages 
(ESOL). She had 2 years experience working in early child- 
hood settings. Rhonda was 39 years old when the study 
began, was a mentor teacher, and had 10 years 7 months 
experience working in early childhood settings. She held an 
associate’s degree in elementary and early childhood educa- 
tion and was also a full-time student working toward a 
bachelor’s degree in early childhood education. She had a 
director’s and a Child Development Associate credential. 
Brenda was 24 years old when the study began and held a 
bachelor’s degree in psychology. She had 5 years 2 months 
experience working in early childhood settings and had no 
teaching certifications or endorsements. Both Natalie and 
Rhonda shared teaching responsibilities with a co-teacher. 
Brenda was the only teacher in her classroom. 

Natalie was the only teacher in the study who reported 
familiarity with the terminology for the components of 
embedded instruction learning trials prior to learning about 
them in the study. None of the teachers indicated they were 
planning for or implementing embedded instruction learn- 
ing trials as part of targeted or individualized instruction for 
children in their classrooms. We conducted observations 
using the Classroom Assessment Scoring System (CLASS; 
Pianta, La Paro, & Hamre, 2008) to characterize the types 
of classroom organization and instructional supports for 
embedded instruction. CLASS scores indicated low-range 
scores on quality domains of classroom organization 
(range = 3.1—3.5) and instructional support (range = 2.1— 
2.5), which include dimensions related to instructional 
learning formats, productivity, and the quality of feedback 
provided to children. 


Child participants. Each teacher nominated a child to partici- 
pate in the study. To be eligible for the study, children had 
to (a) be between the ages of 24 and 60 months when the 
study began, (b) not be transitioning to a new classroom 
during the course of the study, and (c) be identified by the 
teacher as being at-risk for learning challenges. 

All teachers in the study used the Ages and Stages 
Questionnaires, Third Edition (ASQ-3; Squires, Twombly, 
Bricker, & Potter, 2009). Information from these assess- 
ments and from observations of children was used to deter- 
mine whether there were children who needed more targeted 
instruction in a particular developmental domain because 
they were at risk for learning challenges. This information 
was also used to inform children’s eligibility for the present 
study. 

Three children participated in the study: David, Amanda, 
and Allison. None of the children were dual language learn- 
ers. All three children were eligible for participation based 
on ongoing observational assessment that they were at risk 
for learning challenges. David was 57 months old when the 
study began and was White/non-Hispanic. He had been 
attending preschool for 32 months and had been in Natalie’s 


classroom for 12 months. Natalie expressed concerns on the 
ASQ-3 that David had delays in some drawing skills and 
noted from observations that he showed limited engage- 
ment in early literacy activities that involved drawing and 
writing. Amanda was 26 months old when the study began 
and was White/non-Hispanic. She had been attending pre- 
school for 18 months and had been in Rhonda’s classroom 
for 2 months. Rhonda reported Amanda’s expressive com- 
munication during interactions with peers was of concern. 
Allison was 43 months old when the study began and was 
multiracial. She had been attending preschool for 26 months 
and had been in Brenda’s classroom for 3 months. Allison 
was reported by her teacher and parent to have difficulty 
with receptive language skills, including following multi- 
ple-step directions. 


Settings 


Classrooms in the center in which the study was conducted 
were organized in groups of three according to child age. 
Classrooms were arranged in an L-shape around two rest- 
rooms, which were shared among the classrooms. Natalie 
and Brenda were in adjoining classrooms. They followed 
the same daily schedule but conducted scheduled activities 
and instruction independently. There were 14 children in 
Natalie’s classroom and 9 in Brenda’s classroom. Rhonda’s 
classroom adjoined to two other classrooms. There were a 
total of 30 children across the three classrooms, with 9 chil- 
dren in Rhonda’s classroom. Rhonda and one other teacher 
followed the same daily schedule. These two classrooms 
sometimes combined for morning circle, and they always 
combined for learning centers. 


Procedures 


Teachers were recruited via an email announcement sent by 
the center director. Teachers who were interested in partici- 
pating in the study contacted the first author, and eligibility 
was determined based on the inclusion criteria. Each teacher 
communicated with the child’s parent to obtain informed 
consent for his or her participation in the study. 

CLASS (Pianta et al., 2008) observations were con- 
ducted according to the administration protocol by two 
trained observers during the time when teachers were 
obtaining consent for child participants. Inter-observer 
agreement was calculated at the dimension-item level for 
one of the CLASS observations. Inter-observer agreement 
was 86%. 


Pre-intervention. After children were enrolled, each teacher 
completed a child demographic form to indicate child’s 
gender, age, number of months enrolled in preschool, and 
number of months in the teacher’s classroom. We obtained 
permission from both the center and the child’s parents to 
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Table |. Child Learning Target Behaviors and Target Activities. 


Activity duration in min 


Child Initial learning target behavior Final learning target behavior Target activity M (Range) 
David Use a writing tool to make letters Write uppercase letters and Literacy learning center 10 (4-25) 
or drawings with straight, numbers with a model facilitated by Natalie 
curved, or angled markings 
Amanda Use a three-word phrase with at Use a three-word phrase with Learning center activity 12 (5-21) 
least one describing word to at least one describing word facilitated by Rhonda 
make requests, describe objects/ to make requests, describe 
actions, or answer questions objects/actions, or answer 
(e.g., my red ball, he runs fast, | questions (e.g., my red ball, 
want blue) he runs fast, | want blue) 
Allison Complete multi-step transition Complete multi-step transition Transition from lunch 17 (12-27) 


tasks (e.g., brush teeth, 
wash hands, clean up lunch) 
independently 


tasks (e.g., brush teeth, 

wash hands, clean up lunch) 
independently within 3 min of 
starting the task 


to nap 


retain a copy of the child’s ASQ-3 completed prior and most 
proximal to the start of the study. 

Each teacher met with the first author to review informa- 
tion from the ASQ-3 and their observations of the children 
they nominated to participate in the study. Teachers dis- 
cussed their concerns about the children they nominated to 
participate in the study, and the first author observed the 
children to confirm they were eligible to participate in the 
study due to learning challenges. Information provided by 
the teacher and documented in the first author’s observa- 
tions of the children was used to determine an individual- 
ized learning target for the child. The term /earning target 
was defined for the teacher as an instructional goal for the 
child. The learning target specified an observable child 
behavior the teacher wanted the child to demonstrate. After 
the learning target was determined, the teacher and first 
author identified one classroom activity, routine, or transi- 
tion in which the teacher would teach the target behavior 
and which could be videotaped for self-monitoring and data 
collection purposes. Throughout the study, the teacher and 
first author monitored the child’s demonstration of the tar- 
get behavior and modified the child’s learning target if the 
child demonstrated mastery or if the target behavior 
appeared to be too difficult for the child. Each child’s learn- 
ing target was focused on the same developmental domain 
from the ASQ-3 throughout the study. The activity during 
which videotaping occurred did not change. Information 
about the activities and children’s learning targets is shown 
in Table 1. 

During pre-intervention, study data collectors gathered 
video of the teachers implementing learning trials in the tar- 
get activities shown in Table 1, but these videos were not 
shared with the teachers. Teachers did not receive informa- 
tion on the components of embedded instruction learning 
trials nor were they given access to materials used in the 
self-monitoring conditions. 


Video self-monitoring: Condition |. Following pre-interven- 
tion, teachers were given a digital video camera and asked 
to begin self-monitoring their implementation of learning 
trials using a self-monitoring coding form with general 
prompts corresponding to the components of a CLT. The 
Self-Monitoring Coding Form had the following series of 
open-ended questions to help teachers identify and describe 
the components of instructional learning trials: (a) At what 
time in the video was there an intentional learning opportu- 
nity provided or arranged by you? (b) What happened to 
provide an opportunity for the child to practice the learning 
target behavior? (c) Did the target behavior occur? (d) What 
did you do after the target behavior occurred, if anything? 
(e) If the target behavior did not occur, what did you do 
next, if anything? 

At naptime on the first day of this video self-monitoring 
condition, the teacher engaged in a 15- to 20-min orienta- 
tion meeting in which the first author (a) described how to 
operate the digital video camera, (b) provided the teacher 
with Se/f-Monitoring Coding Forms, and (c) described how 
to complete the Se/f-Monitoring Coding Form. This orien- 
tation did not include any training on the components of 
CLTs or opportunities for the teacher to practice coding and 
receive feedback on her use of the coding form. At the end 
of the meeting, the first author gave the teacher her video 
from earlier in the day and instructed her to review the 
video and complete the Self-Monitoring Coding Form 
before the next scheduled data collection. Teachers used 
video collected by study data collectors during the estab- 
lished target activity and the Sel/f-Monitoring Coding Form 
3 to 5 times a week to enter answers to the series of ques- 
tions related to their implementation of embedded instruc- 
tion learning trials. 


Video self-monitoring: Condition Il. Following the first self- 
monitoring condition, teachers received systematic training 
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by the first author about the components of CLTs and began 
using the Embedded Instruction Observation System— 
Teacher Version (EIOS-T; Bishop, Snyder, Crow, Mullen, 
& Embedded Instruction for Early Learning Project, 2011) 
to self-monitor their implementation of embedded instruc- 
tion learning trials from video collected by study data col- 
lectors. The EIOS-T is a system for monitoring 
implementation of CLTs that includes a coding form, cod- 
ing manual, and training materials. The EIOS-T coding 
form has a series of coding prompts with precise terminol- 
ogy for teachers to record the occurrence and accuracy of 
the antecedent, child behavior, and consequence or error 
correction (presented as “extra help”) components of 
instructional learning trials rather than the open-ended 
questions that appeared on the Self-Monitoring Coding 
Form. The EIOS-T coding manual includes information on 
(a) the components of an instructional learning trial, (b) 
how to determine when a trial occurs, and (c) how to deter- 
mine whether each component of the trial is implemented 
with fidelity. The EIOS-T training was a 2-hr, interactive 
training that reviewed and supplemented coding manual 
content. 

Participants received the EIOS-T coding manual and 
participated in the EIOS-T training session before they 
began self-monitoring coding using the EIOS-T. The train- 
ing was delivered by the first author and occurred outside of 
work hours at a time and location convenient for the teacher. 
The first 90 min of the training was devoted to familiarizing 
the teacher with the components of instructional trials and 
EIOS-T coding procedures. For each component of a trial, 
the trainer (a) gave the teacher a definition of the compo- 
nent, (b) explained coding rules to help the teacher identify 
when the component was occurring and whether it was 
implemented with fidelity, (c) showed a video clip of the 
component being implemented correctly and a video clip of 
the component being implemented incorrectly or not at all, 
(d) modeled how to record the occurrence and accuracy of 
the component on the EIOS-T coding form, (e) brain- 
stormed with the teacher what the component might look 
like relative to her child’s target behavior, and (f) answered 
questions the teacher had about coding the component. The 
trainer also discussed how to summarize EJOS-T data and 
use this information to evaluate whether implemented trials 
were CLTs. 

In the final 30 min of the training, the teacher practiced 
coding independently. Each teacher viewed and coded four 
video clips from her pre-intervention sessions. Each teacher 
coded one video clip of herself implementing (a) a trial in 
which the target behavior occurred without extra help and a 
consequence was provided (i.e., CLT), (b) a trial in which 
the target behavior occurred without extra help and no con- 
sequence was provided (i.e., incomplete learning trial), (c) 
a trial in which the target behavior occurred following extra 
help and a consequence was provided (i.e., CLT), and (d) a 


trial in which the target behavior did not occur and no extra 
help was provided (i.e., incomplete learning trial). Each 
teacher also coded three, 1- to 2-min videos of a preschool 
teacher unfamiliar to her implementing multiple trials dur- 
ing a classroom activity or routine. The trainer provided 
immediate feedback about the teachers’ coding of the occur- 
rence and accuracy of learning trial components for each 
trial. Teachers were required to demonstrate at least 80% 
coding accuracy for learning trial occurrences and learning 
trial components on three of the seven practice videos (i.e., 
four pre-intervention video clips and three clips of an unfa- 
miliar teacher) before continuing with the self-monitoring 
intervention. 

After teachers demonstrated 80% coding accuracy on 
three practice videos, they used the videos collected by 
study data collectors and the EIOS-T coding form to self- 
monitor their implementation of learning trials 3 to 5 times 
per week. No additional training on CLT components was 
provided nor did teachers receive feedback about their cod- 
ing accuracy. 


Video self-monitoring: Condition Ill. In the final intervention 
condition, teachers continued to use the EIOS-T to self- 
monitor their implementation of trials, and they received 
ongoing feedback from the first author about their self- 
monitoring accuracy. We used correspondence between the 
teacher’s self-monitoring data and the researcher’s data on 
the occurrence of trials and the components of CLTs imple- 
mented as a measure of self-monitoring accuracy. On the 
first day of this condition, the first author videotaped the 
teacher implementing trials on the child’s identified target 
behavior during the targeted routine or activity. The first 
feedback session was a 30- to 45-min interactive discussion 
that occurred at naptime of the same day. The goal of this 
session was for the teacher and first author to gain consen- 
sus on how the child’s target behavior would be recognized. 
The first author presented an EJOS-T Coding Feedback 
Form, which highlighted the teacher’s coding strengths and 
explained discrepancies between the teacher’s and first 
author’s coding that were due to differences in their defini- 
tions of the target behavior. Feedback was based on com- 
parisons of the teacher’s and first author’s coding from 
sessions in the video self-monitoring II condition. The 
teacher and first author completed a worksheet together to 
reach consensus about how they would identify when the 
target behavior occurred. At the end of the session, the 
teacher was instructed to view the video collected by the 
first author earlier that day and use the EIOS-T coding form 
to self-monitor her implementation of trials and CLTs on the 
revised target behavior. 

For each subsequent session, the first author videotaped 
the teacher and child in the target activity and collected the 
teacher’s EJOS-T Self-Monitoring Coding Form from the 
previous session. After filming, the first author reviewed 
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the teacher’s coding form from the previous session and 
completed the EJOS-T Coding Feedback Form. At naptime 
of the same day, the first author and teacher met for 10 to 20 
min to discuss the teacher’s coding. Every feedback session 
included (a) opportunities for the teacher to ask questions 
about her coding, (b) a presentation of a summary of coding 
agreements and disagreements between the teacher and first 
author, (c) written and verbal positive feedback regarding 
the teacher’s use of the EIOS-T to monitor her implementa- 
tion of trials and CLTs, (d) written and verbal corrective 
feedback regarding the teacher’s use of the EIOS-T to mon- 
itor her implementation of learning trials, and (e) confirma- 
tion of the child’s target behavior. Supportive and corrective 
feedback were provided in a variety of ways, including (a) 
graphs showing changes in the teacher’s coding accuracy, 
(b) reviewing video of the teacher implementing trials and 
discussing coding discrepancies, and (c) modeling EIOS-T 
coding procedures for the teacher. 


Maintenance. A data collector videotaped the teacher | to 3 
times per week during the target activity and provided the 
video to the teacher. Each teacher was given the option to 
self-monitor her implementation of trials using either the 
Self-Monitoring Coding Form or the EIOS-T. No training 
or feedback about self-monitoring accuracy was provided. 


Instrumentation 


Project-developed measures were used to assess the (a) 
fidelity with which the researcher implemented the compo- 
nents of the self-monitoring training and feedback sessions, 
(b) correspondence between teachers’ self-monitoring data 
and the researcher’s data about the occurrence of trials and 
components of CLTs, (c) percentage of CLTs implemented 
by the teacher, and (d) social validity. 


Training and feedback fidelity. Data were collected for at least 
33% of all trainings and feedback sessions. Project-devel- 
oped checklists were used to assess procedural fidelity for 
all intervention conditions. Each checklist outlined the steps 
to be completed in the training session for each condition 
and provided spaces for an observer to indicate whether or 
not each step occurred. A graduate assistant familiar with the 
training components of the study listened to an audio record- 
ing of each training or feedback session and completed the 
corresponding checklist by indicating whether or not each 
step of the protocol occurred. Procedural fidelity was calcu- 
lated by dividing the number of steps completed by the total 
number of steps and multiplying the result by 100. 


Self-monitoring accuracy. We calculated the agreement 
between the teachers’ and first author’s recordings of trial 
occurrences and CLT components. Occurrence agreement 
was defined as the level of agreement between the teacher 


and researcher on the occurrence of trials to elicit the child’s 
target behavior. To examine occurrence agreement, we com- 
pared the time codes recorded by the teacher and researcher 
to indicate the occurrence of a trial. Time codes within 3 s of 
one another were considered agreements. If two time codes 
were close to one another but more than 3 s apart, we 
reviewed the video to determine whether both time codes 
corresponded to the same trial. If both time codes corre- 
sponded to the same trial, these codes were counted as occur- 
rence agreements. Time codes more than 3 s apart that did not 
correspond to the same trial were counted as disagreements. 
To calculate the percentage of occurrence agreement, we 
divided the total number of agreements by the sum of agree- 
ments and disagreements and multiplied the result by 100. 

Component agreement was defined as the extent to 
which the teacher and researcher agreed on the fidelity with 
which each CLT component was implemented. To examine 
component agreement, we compared the antecedent, child 
behavior, consequence, and error correction codes for all 
trial occurrences recorded by both the teacher and researcher. 
The percentage of component agreement was calculated by 
dividing the total number of agreements by the sum of 
agreements and disagreements and multiplied by 100. 
Because the Self-Monitoring Coding Form only provided 
teachers with an opportunity to describe learning trial com- 
ponents but did not include EIOS-T codes, component 
agreement was only calculated for the video self-monitor- 
ing II and video self-monitoring III conditions. 


Implementation of trials and CLTs. We used the EIOS-T cod- 
ing form to record teachers’ occurrence of trials and accu- 
racy of CLTs. The first author was the primary coder for the 
study and coded every video collected. A graduate assistant 
coded 33% of all videos collected in the study for calculat- 
ing inter-observer agreement. Both coders helped develop 
the EIOS-T coding system and so were familiar with the 
components of instructional learning trials. Each coder was 
required to demonstrate 80% inter-observer trial occurrence 
and component agreement on three consecutive training 
videos of teachers providing embedded instruction learning 
trials within the context of ongoing preschool activities 
before coding study video footage. Each training video was 
a minimum of 10 min. 

Throughout the study, the two coders met each time a 
child’s individualized learning target changed to gain con- 
sensus on how an occurrence of the child’s target behavior 
would be recognized for coding. To record the occurrence 
of learning trials, coders played the videotape and docu- 
mented the time code for every intentional antecedent the 
teacher provided or arranged in the environment to elicit the 
child’s target behavior. Coders indicated whether the target 
behavior occurred after the antecedent and whether an 
appropriate consequence or error correction procedure was 
provided. In trials where an error correction procedure was 


176 


Topics in Early Childhood Special Education 35(3) 


necessary to elicit the target behavior, the coder indicated 
whether the target behavior occurred after error correction 
and whether the teacher provided an appropriate conse- 
quence or feedback. 

For each session, coders summarized the percentage of 
CLTs. A trial was considered complete if all teacher-deliv- 
ered or environmentally arranged components (i.e., ante- 
cedent, consequence, error correction) were implemented 
appropriately before and after the targeted child behavior. 
These data were graphed for visual analysis to examine 
teachers’ implementation of CLTs across all experimental 
conditions. The percentage was calculated by dividing the 
number of CLTs by the total number of trials attempted and 
multiplying the result by 100. 


Inter-observer agreement. Occurrence and component 
agreement were examined for 33% of all coded sessions 
(n = 37). Mean occurrence agreement was 87% (range = 
43-100) overall and 92% (range = 71-100), 81% (range = 
50-100), and 89% (range = 43-100) for Natalie, Rhonda, 
and Brenda, respectively. Mean component agreement was 
87% (range = 50-100) overall and 91% (range = 75-100), 
84% (range = 50-100), and 88% (range = 62—100) for Nata- 
lie, Rhonda, and Brenda, respectively. 


Social validity. To assess teacher acceptability of the self- 
monitoring instruments used in the study, we asked teachers 
to complete Intervention Rating Profiles (IRP) for the Self- 
Monitoring Coding Form and for the EIOS-T. The IRPs 
were adapted versions of an available instrument measuring 
the acceptability of behavioral interventions implemented 
by teachers in classroom settings (Martens, Witt, Elliott, & 
Darveaux, 1985). Both the IRP for the Sel/f-Monitoring Cod- 
ing Form (IRP-SM) and the IRP for the EIOS-T (IRP- 
EIOST) are 12-item scales. Teachers responded to items 
regarding (a) the usefulness of the form to evaluate instruc- 
tional practices (v = 3), (b) the value of the form with respect 
to the time needed to use it to self-monitor instruction (v = 
2), (c) practicality of using the form in terms of time and 
technical skill required (v = 2), (d) usefulness of the form for 
evaluating the implementation of instructional learning trials 
(v=4), (e) and overall benefit of the form to teachers (v= 1). 
Responses were recorded on a 6-point Likert-type scale 
ranging from | (strongly disagree) to 6 (strongly agree). The 
IRP-SM was administered at the end of the first self-moni- 
toring intervention condition. The IRP-EIOST was adminis- 
tered at the end of the third self-monitoring intervention 
condition. Teachers were asked not to identify themselves on 
the forms, so their answers could remain anonymous. 


Experimental Design and Data Collection 


We used a multi-component single-subject experimental 
design with probes (Horner & Baer, 1978) across teachers. 


Consistent with multiple probe procedures described by 
Kennedy (2005), data collection occurred intermittently 
during the execution of the experimental series, with at least 
three probes occurring in the same week before a condition 
change. Data collection and analysis procedures are 
described for each phase of the study. 


Pre-intervention. Pre-intervention data collection began 
after the teacher and first author identified an individual- 
ized learning target for the child and occurred during the 
target activity identified by the teacher and first author 
(see Table 1). To determine whether teachers were engag- 
ing in other types of self-monitoring, data collectors 
observed whether teachers (a) had access to or used video 
cameras during the target activities, (b) used checklists or 
other forms to document their implementation of embed- 
ded instruction learning trials, or (c) reported using other 
procedures to self-monitor their implementation of learn- 
ing trials. 

Data collectors used project video cameras to collect 
video data to examine teachers’ implementation of trials 
during the target activities shown in Table 1. On each day of 
videotaping, the data collector (a) confirmed the child’s tar- 
get behavior with the teacher, (b) asked the teacher for 
information about how she planned to provide instruction to 
elicit the target behavior, and (c) videotaped the teacher and 
child during the target activity. Data collection occurred 1 to 
5 times per week. 


Video self-monitoring |. On the first day of this condition, the 
first author used the teacher’s video camera to videotape the 
teacher during the target activity and provided the teacher 
with a copy of her video during the orientation session 
described previously. Occurrence agreement was calculated 
for this session. 

Following the orientation session, each teacher was vid- 
eotaped 3 to 5 times per week using her own video camera. 
The same videotaping procedures outlined for the pre-inter- 
vention condition were used, and the teacher’s Self- 
Monitoring Coding Form was collected for the previous 
session. The teacher was provided with a copy of her video 
to review, so she could self-monitor and record her imple- 
mentation of learning trials. Data from these sessions were 
used to analyze learning trial occurrence agreement and 
teachers’ implementation of CLTs. 


Video self-monitoring II. Data collection for this condition 
began the day following the EIOS-T training. Each teacher 
was videotaped 3 to 5 times per week following the same 
procedures used in the video self-monitoring I condition. 
Data from these sessions were used to analyze trial occur- 
rence and CLT component agreement between the teacher 
and the researcher as well as teachers’ implementation of 
learning trials. 
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Video self-monitoring Ill. Data collection for this condition 
began the day of the first feedback session. Data collection 
and feedback sessions occurred 3 to 5 times per week. 
Video collection procedures were the same as the previous 
two intervention phases, and the data analysis procedures 
were the same as those described for the video self-monitor- 
ing II condition. 


Maintenance. Videotaping continued 1 to 3 times per week 
following the same procedures used in the intervention con- 
ditions. When teachers submitted coding forms during the 
maintenance phase, we calculated the type(s) of agreement 
appropriate for the coding form submitted by the teacher 
(i.e., occurrence and component for EIOS-T, occurrence for 
Self-Monitoring Coding Form). Data from these sessions 
were also used to analyze teachers’ implementation of 
learning trials. 


Results 


Training and Feedback Fidelity 


A fidelity checklist was completed for 50% of all researcher- 
implemented orientation, training, and feedback sessions (n 
= 15). In total, one Self-Monitoring Training Fidelity 
Checklist, two EIOS-T Training Fidelity Checklists, three 
EIOS-T Feedback Orientation Checklists, and nine EIOS-T 
Feedback Checklists were completed. Each step on the 
fidelity checklists was implemented for the orientation to 
the Self-Monitoring Coding Form, the EIOS-T training, and 
the feedback orientation sessions. Across the nine EIOS-T 
feedback sessions, an average of 97% steps were imple- 
mented (range = 87-100). 


Self-Monitoring Accuracy 


There was no evidence that teachers were self-monitoring 
their implementation of embedded instruction learning tri- 
als during pre-intervention; therefore, occurrence and com- 
ponent agreement were not recorded for these sessions. As 
shown in Figure 1, none of the teachers achieved the crite- 
rion of 80% agreement with the researcher’s data until the 
video self-monitoring III condition. Natalie and Brenda 
consistently demonstrated 80% or higher occurrence agree- 
ment with the researcher’s data during this condition. 
Component coding agreement levels increased and became 
less variable for all three teachers in this condition. 
Natalie’s mean occurrence agreement during video self- 
monitoring I, II, and III was 12% (range = 0-20), 62% (range 
= 38-75), and 90% (range = 75-100), respectively. Her 
occurrence agreement varied by approximately 20% within 
in each condition, but she demonstrated increases in occur- 
rence agreement across each condition, with 100% non- 
overlapping data in Conditions II and II] compared with 


Condition I, and only one overlapping data point across 
Conditions III and II. Following her fifth feedback session, 
Natalie’s occurrence agreement with the first author was 
consistently 100%. Similar findings were observed with 
respect to component agreement. Natalie’s mean component 
agreement for Conditions II and III was 66% (range = 40— 
83) and 88% (range = 80-100), respectively, with only one 
non-overlapping data point across conditions. 

Rhonda’s mean occurrence agreement during Conditions 
I, I, and II was 31% (range = 9-50), 36% (range = 9-50), 
and 47% (range = 18-76), respectively. Her occurrence 
agreement was highly variable throughout the study. When 
she began receiving feedback, Rhonda showed some 
improvement in occurrence agreement for specific sessions, 
but these changes were not consistent. This is shown by 
completely overlapping data between Conditions I and II, 
and only three of eight non-overlapping data points across 
Condition HI and Conditions I and II. Rhonda’s mean com- 
ponent agreement for Conditions II and III was 66% (range 
= 33-83) and 83% (range = 69-100), respectively. In the 
final four sessions of the feedback condition, she consis- 
tently showed component agreement between 80% and 
100%. Rhonda was the only teacher to submit a coding 
form during maintenance, and her agreement with the 
researcher’s data approximated the final sessions of the 
feedback condition. 

Brenda’s mean occurrence agreement during Conditions 
I, II, and III was 33% (range = 0-66), 41% (range = 20-63), 
and 90% (range = 78-100), respectively. Her occurrence 
agreement was relatively stable in the beginning of 
Condition I and then varied in the final three sessions of this 
condition. This variability continued throughout Condition 
II. Although her mean occurrence agreement was higher in 
Condition II, all data points in this condition overlapped 
with data points in Condition I. In the feedback condition, 
Brenda’s occurrence agreement increased and became less 
variable, with 100% non-overlapping data between this 
condition and Conditions I and II. Brenda’s mean compo- 
nent agreement during the video self-monitoring II and III 
conditions was 67% (range = 39-89) and 82% (range = 
61-83), respectively. Component agreement for all except 
two of her coded sessions in the feedback condition was 
above 80%, compared with only three sessions above 80% 
in Condition II. Although there was less variability in her 
component agreement during Condition III, only one of 
seven data points in this condition was non-overlapping 
with those in Condition II. 


Teachers’ Implementation of CLTs 


Changes in percentage of CLTs are shown in Figure 2. All 
three teachers demonstrated increases in the percentage of 
CLTs during at least one of the video self-monitoring condi- 
tions. Natalie’s mean percentage of CLIs implemented 
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Figure I. Correspondence between teachers’ self-monitoring data and researcher’s data on the occurrence and components of 


embedded instruction learning trials. 
Note. VSM = video self-monitoring. 


during Conditions I, II, and III was 19% (SD = 22.3), 72.4% 
(SD = 11.5), and 98% (SD = 3.6), respectively. All of the data 
points in Condition I overlapped with those in pre-interven- 
tion, and percentage of CLTs decreased to 0 just before 
receiving training on the EIOS-T. Positive changes in 
Natalie’s implementation of CLTs were immediate following 
EIOS-T training. Her mean percentage of CLTs was substan- 
tially higher in Conditions II and III than in pre-intervention 
or Condition I, with no overlapping data points between 


pre-intervention and each of these two conditions. Natalie’s 
implementation of CLTs was highest and most stable after 
she began receiving feedback on her EIOS-T coding accu- 
racy. Her implementation of CLTs became slightly more vari- 
able when self-monitoring stopped, but she generally 
maintained implementing a similar percentage of CLTs as in 
Condition III. 

David’s target behavior was modified 3 times during 
the study (see Figure 2). Modifications were made because 
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Figure 2. Changes in percentage of CLTs across self-monitoring conditions. 
Note. TM indicates points at which the child’s target behavior was modified. CLT = complete learning trials; VSM = video self-monitoring; TM = target 


modified. 


he consistently demonstrated the previous target behavior. 
The modifications involved increasing the complexity of 
the target skills (see Table 1). Natalie’s implementation of 
CLTs around the time of the first target behavior modifica- 
tion in Condition II increased compared with the previous 
two sessions but was within the range of observed percent- 
ages (57%-88%) of CLTs observed in Condition II. Her 
implementation of CLTs around the time of the second tar- 
get behavior modification in Condition II was higher than 
any previous sessions (95%). Subsequent observations 
showed variability around this level of implementation 
(91%-100%), and her implementation was consistent with 
previous sessions at the time of the third learning target 
modification. 


Rhonda’s mean percentage of CLTs during Conditions I, 
II, and III was 54.7% (SD = 9.7), 55.9% (SD = 15.7), and 
74.9% (SD = 12.7), respectively. Her implementation of 
CLTs was variable within and across conditions. Although 
her mean percentage of CLTs was higher in Conditions I, I, 
and III compared with pre-intervention, the only condition 
in which there was 100% non-overlapping data compared 
with pre-intervention was Condition II. During mainte- 
nance, Rhonda had similar percentages of CLTs across ses- 
sions to those observed in each of the video self-monitoring 
conditions, and on average, these levels were higher than 
those in pre-intervention. 

Brenda’s mean percentage of CLTs during Conditions I, 
II, and III was 10.4% (SD = 14.5), 28.7% (SD = 22.2), and 
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75.7% (SD = 13.7), respectively. Her implementation of 
CLTs varied within and across conditions. Nevertheless, she 
consistently demonstrated the highest percentages of CLTs 
during video self-monitoring with feedback (Condition III), 
as shown by six of seven non-overlapping data points across 
this condition and pre-intervention. In addition, she had 
only one non-overlapping data point between Condition III 
and Condition II and completely overlapping data between 
Condition II and pre-intervention. Brenda’s implementation 
of CLTs was generally higher and less variable in mainte- 
nance than in pre-intervention or Conditions I and II, but 
she did not continue to implement CLTs with the same level 
of accuracy as she did during video self-monitoring with 
feedback. 

Allison’s target behavior was modified 4 times during 
the study (see Figure 2). Modifications involved breaking 
down the number of steps in the transition task she was 
required to complete and the duration in which she was 
expected to complete them. Brenda’s implementation of 
CLTs varied around the time of target behavior modifica- 
tions. However, there was no apparent deviation in variabil- 
ity of implementation compared with other observations in 
the conditions in which modifications were made (pre- 
intervention = 0%-63%, Condition II = 67%, Condition 
Il = 60%-100%). 


Social Validity 


Teacher ratings on the IRP-SM and the IRP-EIOST showed 
teachers found both self-monitoring instruments accept- 
able. The overall mean rating for the Self-Monitoring 
Coding Form was 4.8 (range = 4.5—5.2), and for the EIOS-T, 
it was 4.8 (range = 3.8-5.4). Average ratings for items on 
both IRP instruments were nearly identical across teachers. 
Average IRP ratings for the item pertaining to the overall 
benefit of the Self-Monitoring Coding Form were slightly 
higher than ratings for the overall benefit of the EIOS-T 
(5.3 and 4.7, respectively). 


Discussion 


Findings from the present study show the combination of 
training, self-monitoring forms that explicitly defined CLTs, 
and external feedback resulted in more accurate video self- 
monitoring for two of the three teachers in the study. As 
teachers’ accuracy of video self-monitoring increased, 
researcher-observed implementation of CLTs increased. The 
training provided in the video self-monitoring II condition 
had components similar to the training provided in previous 
studies examining relationships between self-monitoring 
and teachers’ implementation of instructional learning trials 
(e.g., Belfiore et al., 2008; Lylo & Lee, 2013). These compo- 
nents include definitions of behaviors teachers will self- 
monitor, modeling self-monitoring procedures, feedback 


from a trainer on self-monitoring accuracy, and required 
demonstration of self-monitoring accuracy before engaging 
in independent self-monitoring. 

Our data show that teachers met the required self-moni- 
toring accuracy criteria during training but demonstrated 
modest and variable levels of self-monitoring accuracy dur- 
ing Condition II. Their accuracy generally improved when 
they received feedback in video self-monitoring Condition 
Ill. Our findings differ from those of previous studies, 
which reported teachers reliably met self-monitoring accu- 
racy criteria in the initial training and generally continued to 
self-monitor at or above the established criteria without 
receiving additional supports. One explanation for this find- 
ing might be that in previous studies, teachers monitored 
their implementation of discrete or massed trials during 
structured direct instruction activities. It is possible teachers 
in the present study required more support to self-monitor 
reliably because self-monitoring occurred in relatively 
unstructured activities, and learning trials were embedded 
throughout the activities. In addition, teachers in the present 
study might not have been as familiar with the components 
of instructional learning trials before participating in the 
study as those in previous studies. 

Our findings also show positive relationships between 
video self-monitoring with training and feedback and the 
fidelity with which teachers implemented CLTs. Changes in 
the percentages of CLTs implemented across conditions in 
the present study were similar to those reported in previous 
studies (e.g., Belfiore et al., 2008; Lylo & Lee, 2013; 
Pelletier et al., 2010), but the intensity of support provided 
from an external agent to achieve these changes differed. In 
addition, the magnitude of change varied across teachers, 
which suggests a need for additional research to determine 
under what conditions and for whom self-coaching, includ- 
ing self-monitoring, might be most effective. For example, 
the primary foci of the training and feedback in the present 
study were to identify the occurrence and evaluate the accu- 
racy of instructional trials rather than increase the number 
of trials implemented. Video self-monitoring with training 
and feedback was most effective for Natalie, who had the 
highest level of training, which included a bachelor’s degree 
in special education. Natalie also reported familiarity with 
the components of embedded instruction learning trials 
before intervention began. It is possible that findings for 
Rhonda and Brenda might have been more robust if they 
had received training on embedded instruction learning tri- 
als before implementing the self-monitoring interventions 
evaluated in the present study. 

During maintenance, only one teacher showed CLT 
implementation percentages similar to those shown when 
she was receiving feedback on her self-monitoring accu- 
racy. Belfiore and colleagues (2008) observed similar trends 
in teachers’ implementation of discrete learning trials, but 
when the researchers re-initiated the treatment phase (1.e., 
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use of self-monitoring forms without additional training), 
implementation levels increased. These findings suggest a 
potential need for longer periods of supported self-monitor- 
ing. In addition, data from the present study suggest a 
potential need for occasional “booster” sessions, in which 
teachers receive feedback on their self-monitoring accu- 
racy. Additional research is needed to examine the relative 
effects of different durations of supported self-monitoring 
on teachers’ instructional practices and to identify the fre- 
quency and types of supports, including feedback from an 
external agent, needed to sustain these effects. 

Although teachers generally found both self-monitor- 
ing coding forms acceptable, the maintenance data show 
they did not continue to self-monitor with either tool. The 
implementation data show use of the coding form with 
explicit descriptions of learning trial components was 
more effective at increasing the percentage of CLTs than 
the general coding form. Teachers did not continue to use 
this form, however, to self-monitor their implementation 
of trials. This finding suggests a need for external sup- 
ports, such as reminders or incentives to continue 
self-monitoring. 

Two primary limitations of the present study are noted. 
First, we did not collect data about whether teachers’ self- 
monitoring generalized to other activities or targeted child 
behaviors. Another limitation is that teachers did not col- 
lect their own video data. We provided teachers with per- 
sonal digital video cameras to collect their own data, but 
conditions in their classrooms (e.g., staff—child ratios, 
room arrangements) prevented them from doing so. Future 
studies could explore feasible ways for teachers to gather 
video for use in self-monitoring and identify program- 
matic and infrastructure supports that are necessary to 
self-monitoring. 

Findings from the present study contribute toward under- 
standing which training and coaching components are most 
likely to result in accurate self-monitoring and improved 
implementation of instructional practices. Self-monitoring, 
as used in the present study, included several key compo- 
nents of effective coaching (i.e., defined set of teaching 
practices, self-observation, self-evaluation) identified by 
Snyder and colleagues (2012). Face-to-face expert coach- 
ing might not be feasible or necessary to implement with all 
teachers. Alternative coaching formats, such as self-coach- 
ing, might be efficient and effective to support practice 
implementation for some teachers under certain conditions, 
provided the key components of effective coaching are used 
(NCQTL, 2014). This includes identifying the self-monitor- 
ing practices most important for implementing self-coach- 
ing. In the context of implementation science, continuing to 
explore the utility and effectiveness of self-monitoring as a 
competency driver of evidence-based practice implementa- 
tion seems warranted. 
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